There exist many systems whose activities need to be monitored or recorded. Such systems vary vastly in scale, and might include a computer processor on one hand, or a warehouse or international distribution chain on the other. These systems are commonly arranged to produce metrics representing the various activities of the system which may then be monitored or recorded. It is a fact that the data represented by these metrics can be extremely copious, regardless of the quantity of useful information they comprise. It is common to regularly sample the information provided by the system under inspection, in order to gauge its activity. The choice of sampling rate is potentially problematic however, since if set too low, so that the metrics are only occasionally sampled, brief spikes or dips in measured values may be missed altogether. It is therefore common practice to set a very high sampling rate. This, on the other hand, whilst ensuring that no information is lost, can place a substantial burden on the monitoring or recording system.
In fact it has been observed that once a monitoring regime is introduced its results are generally trusted without further analysis, regardless of whether the sampling rate is in fact optimal. It is virtually unknown for the users of the system to adjust the sampling rate after the system is first set up.
US20040186685A1 provides a method of optimizing a sampling period for a system having at least one measurable system parameter z, including the step of calculating a probability distribution function f(Tz,x). The time Tz,x is a first time that the measurable system parameter z will reach a predetermined system threshold x, given level z. assumes there is a threshold and that the system is not interested in samples that are below this threshold. This method can only be used if the monitoring is threshold based limiting the fields in which it can be uses, and the value of the data generated.
Accordingly, one of the objects of the present invention is to provide a method of processing an arbitrary metric stream which overcomes at least some of the problems associated with the prior art methods.